Misspecification in Inverse Reinforcement Learning

The aim of Inverse Reinforcement Learning (IRL) is to infer a reward function R from policy pi. To do this, we need model how pi relates R. In the current literature, most common models are optimality, Boltzmann rationality, and causal entropy maximisation. One primary motivations behind IRL human preferences behaviour. However, true relationship between behaviour much more complex than any currently used in IRL. This means that they misspecified, which raises worry might lead unsound inferences if applied real-world data. this paper, provide mathematical analysis robust different misspecification, answer precisely demonstrator may differ each standard before leads faulty about We also introduce framework for reasoning misspecification IRL, together with formal tools can be easily derive robustness new models.